Managing Vendor Relations: A Case Study of Two HPC Network Issues

High performance computing requires fast networks to move large amounts of data between compute nodes and disk storage. For a variety of reasons, however, the fast speeds achieved by bleeding edge network technology tend to come along with a higher risk of system failure. When designing and building high end computing systems using the fastest available network equipment, a good relationship with the network vendor becomes absolutely crucial for success. Through 2008 and 2009, the Argonne Leadership Computing Facility (ALCF) faced two serious challenges with the high-speed Myricom data network at the center of its production supercomputers. By studying these challenges and how we worked with Myricom to resolve them, we establish a five-step method for vendorassisted problem resolution that system and network administrators can use to improve their own vendor relations and time to problem resolution. Furthermore, we include specific examples that can help Myricom network administrators study and debug their own networks.